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Some  Comparisons  of  Mathematics  Achievement  in  UICSM  and 
Non-UICSM  Classes  in  Inglewood,    California 

Maurice  M.    Tatsuoka  and  Robert  E.    Comley* 

Abstract 

Three  statistical  analyses  were  performed  on  test 
score  data  for  students  from  Inglewood,    California:     (1) 
a  matched  pairs  design    (Z)    a  covariance  analysis,    and 
(3)    an  item  analysis  in  terms  of  difficulty  indices.     Forty 
matched-pairs  of  UICSM  and  non-UICSM  students  were 
formed  on  three  variables  and  the  two  groups  were  compared 
on  the  Cooperative  Elementary  Algebra  and  Coop  Math, 
Algebra    1,     tests.      No  significant  differences  were  found  in 
test  score  means.      The  covariance  analyses  resulted  in 
statistical  adjustments  on  six  and  five  variables  for  junior 
and  senior  high  school  grbups  of  students   respectively. 
The  same  two  criterion  variables  were  used  for  both  the 
junior  high  and  senior  high  groups  and  significant  differences 
were  found  in  both  groups  when  adjustment  was  not  made  for 
teacher  ratings  by  school  principals.      The  results  showed 
significantly  higher  achievement  by  the  UICSM  students  on 
both  the  Coop  Elementary  Algebra  and  Coop  Math  exam- 
inations.     The  item,  analysis  of  the  two  criterion  tests 
showed  significantly  better  achievement  by  UICSM  students 
on  ten  items  from  the  two  tests;  while  non-UICSM  students 
excelled  in  four  items  at  a  significant  level. 

The  studies  described  in  this  report  were  initiated  by  the  research  section 
of  UICSM  at  the  request  of  Dr.    Quentin  R.    Bryan,    Coordinator  of  Special 
Projects  for  the  Inglewood  Unified  School  District.      These  studies  consisted 
of  tests  administered  by  Inglewood  teachers  but  selected  and  scheduled  in 
close  collaboration  with  the  senior  author,    who  visited  the  school  system  and 
maintained  close  contact  with  Dr.    Bryan  and  his  staff.      This   request  for  help 
offered  an  opportunity  for  the  project  to  obtain  a   "reading"  on  the  level  of 
achievement  in  a  school  system  that  had  newly  adopted  the  UICSM  curriculum. 

The  Inglewood  Unified  School  District  includes  two  junior  high  schools 
(7-8)    and  two  senior  high  schools    (9-12).     Five  classes  of  superior    8th    grade 
students,    one  class  of  superior    9th    grade  students,    and  one  class  of  "randomly" 
selected    9th  grade  students  used  UICSM  materials  as  their  course  of  study. 
Median  intelligence  scores  for  students  in  this  district  have  consistently  been 
above  the  national  norm.      Placements  in  the  junior  high  school  experimental 


*The  authors  wish  especially  to  acknowledge  the  assistance  of  Judith  Boyle 
who  performed  many  of  the  calculations. 


tions  were  made  on  the  basis  of  the  California  Algebra  Aptitude  Test  (CAA), 
.ulministered  in  the  spring  of    1962,    and  the  high  school  students  were  selected 
on  the  basis  of  past  performance  in  mathematics.     A  total  of   209    pupils  were 
enrolled  in  these  seven    UICSM    sections.     Five  of  these  sections  (those  at  the 
8th    grade  level)  had  studied  only  the  first  ten  pages  in  Unit   4,    at  the  time  the 
two  Coop  algebra  tests  were  administered.     The  other  two  experimental  sections 
(9th   grade)  had  completed  Unit   4.     This  means  that  approximately    150    out  of 
the    209    UICSM    students  in  these  sections  completed  only  about  three-fourths  of 
the  First  Course  at  the  time  of  testing. 

Three  studies  have  been  completed  since  the  inception  of  this  collaboration 
between  the  Inglewood  schools  and   UICSM.      These  include  a  comparison  by  a 
matched-pairs  design,    a  total  group  comparison  in  which  covariance  analysis 
provided  statistical  control,    and  an  item  analysis  of  the  tests  which  obtained 
difficulty  indices  for  the    UICSM   experimental  (E)  group  and  the  comparison  (C) 
group  on  each  item. 


A  Comparison  of  Matched  Groups 

Forty  first-year    UICSM    students  (9th    grade)  were  matched  with  forty  non- 
UICSM    9th-grade  students  of  first  year  algebra  on  three  variables:     (1)    CTMM-L, 
(2)    CTMM-NL.i    and    (3)    STEP-Math.2    The  degree  of  matching  may  be  noted  in 
Table    1.     At  the  end  of  the  year,    scores  on  the  Cooperative  Elementary  Algebra 
Test,    Form    T(1950),    and  the  Cooperative  Mathematics  Test,    Algebra   I,    Form 
A(1962)    were  obtained  for  thirty-two  of  the  forty  pairs.     The  use  of  standardized 
tests  of  this  character  as  criterion  variables  for  comparison  of  achievements 
made  by  the  two  groups  were  justified  by  the  fact  that  the    UICSM    course  has  the 
same  general  goals  as  traditional  algebra. 


iCTMM:     California  Test  of  Mental  Maturity    (1957);   (L,    language;     NL, 
non-language)  published  by  the  California  Test  Bureau. 

?STEP-Math:  Sequential  Tests  of  Educational  Progress  —  Mathematics 
(1957)  published  by  the  Cooperative  Test  Division,  Educational  Testing 
Se rvice . 


Table    1.     Mean  differences  on  three  control  variables  and  two  criterion 

variables  for  matched  pairs  of   UICSM   and  comparison  students, 

Control  Variables  Criterion  variables 


.46 

.32 

1.83 

1.04 

.69 

1.  56 

1.  13 

-0.99 

40 

40 

32* 

32 

STEP       Coop  Coop 

CTMM-L  CTMM-NL      Math        Algebra        Math 

Mean  differ- 
ence   (E  -  C)  t  0  .45  .50  2.06        -1.03 

S.    E.    of  mean 

difference    (E  -   C)  .80 

t  0 

No.    of  pairs  40 

*Criterion  scores  were  not  available  for  all   40    pairs  of  students, 
t  Hence,    positive  differences  are  in  favor  of  UICSM. 

As  shown  in  Table    1,     the  results  were  rather  equivocal  and  in  a  strange 
direction.      The  UICSM  group  did  somewhat  better  than  its  counterpart  on  the 
Coop  Algebra  test,    but  the  difference  was  not  statistically  significant  as  determined 
by  a  one-tailed  t-test  at  the    5%   level.      The  comparison  group  on  the  other  hand 
had  a  non-  significantly  higher  mean  on  the  Coop  Math,    Algebra   I,     test.     We  are 
not  entitled  to  draw  any  conclusions.      However,    we  should  note  that  the  opposite 
finding  might  well  have  been  expected,    since  the  Coop  Algebra  test  is  heavy 
on  applications  which  are  less   strongly  emphasized  in  UICSM  first  course 
than  in  traditional  algebra  courses.      On  the  other  hand  the  new  Coop  Math, 
Algebra   I,     test  contains  a  number  of  "modern  mathematics"  items,    e.g.,     29, 
33,     3  5,     36,     so  one  might  have  expected  UICSM  students  to  perform  better  on 
it. 


A    comparison  of  all  UICSM  students  vs.    all  non-UICSM  algebra  students 

A  more  extensive  analysis  of  the  data  on  Inglewood    8th   and   9th    grade   students 
was  carried  out,    using  covariance  techniques  to  apply  statistical  controls.      The 
junior  high  school  group    (8th    grade)    contained    128    UICSM  students    (E)    and   38 
Algebra    I    students    (C).       The  text  used  by  these  non-UICSM  students  was 
Algebra  One,     Hayden  and  Finon    (1961).  .     Covariance  adjustments  were  made  for 
the  control  variables  SCAT-V,    SCAT-Q,     CAA,    STEP-Math,    sex,    and  teacher 


rating  as  made  by  the  principals  of  the  schools.3    An  unrestricted  linear  hypothesis 
model  (using  separate  group  regression  weights)  was  used  instead  of  the  standard 
covariance  analysis.4     Data  relevant  to  the  distributions  of  scores  on  those  tests 
are  given  in  Table    2. 


Table    2.         Means  and  standard  deviations  of  test     scores  for 
the  UICSM-and  non-UICSM  groups  of   8th   grade 
students. 


UICSM 

Non-UICSM 

Mean 

S.    D. 

Mean 

S.    D 

SCAT-V 

Z8Z. 30 

7.  78 

Z8Z. 55 

8.81 

SCAT-Q 

Z96. ZZ 

8.  19 

Z97.  39 

6.94 

CAA 

64.  73 

11.  35 

64.87 

8.67 

STEP-Math 

Z79.  ZZ 

7.96 

Z80. 68 

6.  19 

N^ 

=    128 

N„ 

=    38 

The  criterion  variables  were  the  Coop  Algebra  Test    (1950)    and  the  Coop 
Math,   Algebra   I,     Test    (1962).     The  UICSM  group  mean  scores  exceeded  the 
corresponding  mean  scores  of  the  non-UICSM  group,    and  the  differences  were 
statistically  significant  after  covariance  adjustments  were  made,    if  teacher 
rating  was  excluded,    as  is  shown  in  Table   4. 

The  senior  high  school  group  contained    38    UICSM  students    (E)    and    171 
non-UICSM  students    (C).     The  control  variables,    for  which  covariance 
adjustments  were  made,    include  STEP-Math,    CTMM-L,     CTMM-NL,     sex, 
and  a  teacher  rating  made  by  school  principals.     Means  and  standard  deviations 
of  scores  on  the  control  variables  are  given  in  Table    3. 


SCAT: 


CAA: 


Cooperative  School  and  College  Ability  Tests    (1955) 

(Q,     quantitative;    V,    verbal)  published  by  the  educational 

Testing  Service. 

California  Algebra  Aptitude    Test   (1958)    published  by  the  American 
Guidance  Service,    Im  . 


We  are  indebted  to  Dr.    Frank  Watson,   now  at  the  Cancer  Research  Hospital, 
Columbia,    Missouri,    for  making  available  to  us  his  then  unpublished  computer 
program  for  carrying  out  the  test. 


Table    3.        Means  and  standard  deviations  of  scores  of  UICSM-and 
non-UICSM    9th    grade  students  on  the  control  tests. 


STEP-Math 

CTMM-L 
CTMM-NL 


UICSM 

Mean  S.    D. 

284.92  9.46 

46.03  7.42 

33.37  5.13 


N„    =    38 
E 


Non-UICSM 
Mean  S.    D. 

280.  15  9.06 

40.96  8.32 

30.80  5.27 


Nc    =    171 


The  criterion  variables  were  Coop  Algebra  Test    (1950)    and  the  Coop  Math, 
Algebra   I,     Test    (1962).     Again  the  means  of  the  scores  for  the  UICSM  students 
exceeded  the  corresponding  means  of  the  control  group;  but  after  covariance 
adjustments  for  all  of  the  control  variables,    the  differences  were  not 
statistically  significant.      However',    when  covariance  adjustment  for  teacher 
rating  was  omitted,    the  adjusted  difference  in  mean  scores  on  the  Coop  algebra 
test  was    10.  2    and  was  statistically  significant  at  the    5%   level.      The  results  are 
recorded  in  the  summary  table  below.     It  is  interesting  that  the  significant 
differences  occurred  when  covariance  adjustments  were  not  made  on  teacher 
rating.      This  suggests  that  the  superior  performance  of  UICSM  students  may  be 
due  to  their  having  superior  teachers.      Thus,    either  due  to  the  self- selection 
factor  (superior  teachers  wanting  to  teach  UICSM)    or  to  their  having  profited  from 
the  UICSM  Summer  Institute  training  (or  both),    it  seems  that  UICSM  teachers  do 
get  a  higher  rating  by  principals,    and  that  their  superiority  as  teachers  has  a 
positive  effect  on  student  performance. 


Table    4.         Summary  of  Inglewood  Results 
Junior  High 

E    =    UICSM  (N    =    128) 

C    =   Algebra    1  (N   =    38) 


Criterion 
Variable 


Group  Means 
E  C 


Significance  at    5%   level  when 
covariance  adjustments  were  made  on: 
All  control  vars.  *    All  but  teacher  rating 


Coop.    Algebra     22.4         19.9  Not  signif. 

Coop.    Math 25.  4        23.  5 Not  signif. 


Signif. 
Signif. 


^Control  variables: 


SCAT-V,    SCAT-Q 

CAA,    STEP-Math 
Sex,    Teacher  Rating 


II        Senior  High 

E    =    UICSM 

C    -    Traditional 


Criterion 
Variable 


Group  Means 
E  C 


(N   =    38) 

(N    -    171) 

Significance  at    5%   level  when 
covariance  adjustments  were  made  on: 

All  control  vars.  :':    All  but  teacher  rating 


Coop.    Algebra         34.  5       24.  5  Not  signif. 

Coop.    Math 30.0       25.8 Not  signif. 


Signif. 
Not  signif. 


^Control  variables: 


STEP  Math 

CTMM-L,     CTMM-NL 
Sex,    Teacher  Rating 


In  order  to  permit  some  comparison  of  the  students  in  Inglewood  schools 
with  students  in  the  older  pilot  and  participating  schools,    a  summary  of  data  on 
the  Coop  Algebra  examination  for  students  who  began  study  of  UICSM  Course    I 
in    1958    and    1959    is  included    (Table    5).     A  detailed  discussion  related  to  these 
data  has  been  given  by  Tatsuoka  and  Easley    (1963). 


Table    5.    Coop  Algebra  means  and  standard  deviations  of  raw  scores  for  some 
groups  of  students  in  UICSM  Course    I    and  comparison  groups  in    1958 
and    1959. 

Non-UICSM 


Dates  of 

UICSM 

Testing 

Means 

S.  D.  * 

N 

May    1959 

28.  58 

8 

118 

Sept.  -  Dec. 

,     1959 

26.  72 

9 

270 

May    I960 

11.  70 

12 

574 

Oct.  -Nov. 

I960 

27.  91 

9 

382 

Jan.     1961 

26.  18 

10 

135 

May    1960- 

Mar.     1961 

28.89 

1  1 

226 

Means 

S.  D.  * 

N 

22.  35 

12 

515 

20.  75 

11 

161 

:-  Approximations  of  standard  deviations  of  raw  scores  from  standard  deviations 
of  seal i-tl  scores. 

It  may  be  noted  that  the  Inglewood  Senior  High  UICSM  students  had  a  higher 
mean  score  on  the  Coop  Algebra  examination  than  the  students  in  these  compari- 
son groups. 


Comparison  of  item  difficulty  indices  for  UICSM 
and  non-UICSM  students  in  Inglewood,    California 

A  third  comparison  of  UICSM  and  non-UICSM  students  at  Inglewood  was 
made  with  respect  to  relative  success  on  individual  items  on  the  Coop  Algegra 
and  Coop  Math  tests.      The  percentage  of  students  correctly  answering  each  item 
of  the  two  criterion  tests  was  computed.     Such  percentages  are  customarily 
called  difficulty  indices.      The  smaller  the  index,    the  greater  is  the  difficulty 
of  the  item,    i.  e.  ,    the   smaller  the  number  of  students  answering  the  item 
correctly.      The  calculations  were  based  on  answers  by   209   UICSM  students  and 
249   non-UICSM  students  on  the  Coop  Algebra  test,    and    187   UICSM  and    249 
non-UICSM  students  on  the  Coop  Math  test. 

Some  differences  in  success  for  the  various  test  items  were  noted.      Those 
items  for  which  differences  in  difficulty  indices  for  the  experimental  and 
control  groups  were  found  to  exceed  or  equal    0.  20    (a  significant  difference 
at  the    5%   level)  are  listed  below. 

A.      Diff.    Ind.  —   Diff.     Ind.  >    0.  20    (The  percentage  of  UICSM  students 


correctly  answering  an  item  exceeded,    by    20    or  more,    the  percentage  of 
non-UICSM  students  who  got  the  item  right. ) 

1.  Coop  Algebra  Test 

UICSM  students  did  better  on  items    31,     35,     and    37    on  Part  I   and 
items    10,     11,     and    16.  on  Part   II   of  the  Cooperative  Algebra  test,    Form  T 
(1950),     published  by  the  Educational  Testing  Service.      The  first  group  of 
these  items  deals  with  combining  and  simplifying  algebraic  fractions  and 
the  second  group  deals  with  the  general  concept  of  defining  a  new  quantity 
as  the  quotient  of  two  other  quantities,    e.  g.  ,    speed  as  the  ratio  of  distance 
and  time,    trigonometric  functions  defined  as  ratios  of  particular  sides  of 
a  triangle,    and  cost  per  article  when  given  the  cost  of  many  articles. 

2.  Coop  Math  Test 

UICSM  students  did  significantly  better  on  items    4,     27,     29,     and    38    of 
The  Cooperative  Mathematics  Test,    Algebra   I,     Form    A    (1962)    which  is  also 
published  by  ETS.      The  four  items  listed  above  treat  knowledge  of  algebraic 
fractions,    non-divisibility  by  zero,    inequalities,    and  recognizing  that  a  given 
linear  equation  has  no  solution.      These  are  all  topics  on  which  UICSM 
materials  place  greater  stress  than  conventional  texts  do. 

B.      Diff.     Ind.     _    —    Diff.    Ind.     ^    >    0.  20 
—       C E    ~ 

1.       Coop  Algebra  Test 

The  comparison  group  did  better  than  UICSM  students  on  items    9 
and    16    of  Part  I   of  the  test.      These  items  deal  with  the  laws  of  exponents 
and  graphical  solutions  of  linear  equations.      (As  of  the  time  of  testing,    the 
UICSM  students  had  not  studied  the  laws  of  exponents  or  two-dimensional 
graphs  of  solution  sets.  ) 


2.      Coop  Math    lest 

Items    7    and    9    of  this  test  were  answered  better  by  non-UICSM  students. 
These  items  test  understanding  of  the  definition  of  coefficient  and  the  laws 
dI  exponents.      Again  these  topics  had  not  been  studied  by  UICSM  students 
at  the  time  of  testing. 


A  principal-axis  factor  analysis  is  presently  being  carried  on  in  an  attempt 
to  classify  items  into  several  groups  which  seem  to  depend  on  different  skills 
and  knowledges.      However,    no  conclusions  have  been  reached  at  this  point. 
Item  analysis  is  also  being  carried  on  in  connection  with  a  UICSM  elementary 
algebra  inventory  which  covers  many  objectives  not  covered  in  the  two  Coop 
tests. 
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